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Abstract 

Smoothing causal linear time-invariant filters are studied for continuous time processes. 
The paper suggests a family of causal filters with almost exponential damping of the energy on 
the higher frequencies. These filters are sub-ideal meaning that a faster decay of the frequency 
response would lead to the loss of causality. 
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1 Introduction 

The paper studies smoothing filters for continuous time processes. The consideration is restricted 
by the causal continuous time linear time-invariant filters (LTI filters), i.e. linear filters represented 
as convolution integrals over the historical data. These filters are used in dynamic smoothing, when 
the future values of the process are not available. 
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In the frequency domain, smoothing means reduction of the energy on the higher frequencies. 
In particular, an ideal low-pass filter is a smoothing filter. However, this filter is not causal, i.e., it 
requires the future value of the process. Moreover, a filter with exponential decay of the frequency 
response also cannot be causal IS. It follows from the fact that a sufficient rate of decay of 
energy on higher frequencies implies some predictability of the processes; on the other hand, a 
causal filter cannot transform a general kind of a process into a predictable process. The classical 
result is Nyquist-Shannon-Kotelnikov interpolation theorem that implies that if a process is band- 
limited then it is predictable (see, e.g., HI- 131,1151, lH-ini, |[T5l-|[l8l). Recently, it was found that 
processes with exponential decay of energy on the higher frequencies are weakly predictable on a 
finite time horizon lH. 

We suggest a family of causal smoothing filters with "almost" exponential rate of damping 
the energy on the higher frequencies and with the frequency response that can be selected to 
approximate the real unity uniformly on an arbitrarily large interval. These filters are sub-ideal in 
the sense that their effectiveness in the damping of higher frequencies cannot be exceeded; a faster 
decay of the frequency response is not possible for causal filters. This is because this family of 
causal filters approximates the exponential decay rate of a reference set of non-causal filters O. 

2 Problem setting 

Let x{t) be a continuous-time process, t G R. The output of a linear filter is the process 




where /i : R — R is a given impulse response function. 



If h{t) = for t < 0, then the output of the corresponding filter is 




■t 



2 



In this case, the filter and the impulse response function are said to be causal. The output of a 
causal filter at time t can be calculated using only past historical values x(T)|^<t of the currently 
observable continuous-time input process. 

The goal is to approximate x{t) by a smooth filtered process y{t) via selection of an appropri- 
ate causal impulse response function h{-). 

We are looking for families of the causal smoothing impulse response functions h{-) satisfying 
the following conditions. 

(A) The outputs y(-) approximate processes x(-); the arbitrarily close approximation can be 
achieved by selection of an appropriate impulse response from the family. 

(B) For processes x(-) G L2(R), the outputs y(-) are infinitely differentiable functions. On the 
higher frequencies, the frequency response of the filter is as small as possible, to achieve the 
most effective damping of the energy on the higher frequencies of x. 

(C) The effectiveness of this family in the damping of the higher frequencies cannot be ex- 
ceeded; any faster decay of the frequency response would lead to the loss of causality. 

(D) The effectiveness of this family in the damping of the higher frequencies approximates the 
effectiveness of some reference family of non-causal smoothing filters with a reasonably 
fast decay of the frequency response. 

Note that it is not a trivial task to satisfy Conditions (C)-(D). For instance, consider a family of 
low-pass filters with increasing pass interval [—A, A], where A > 1. Clearly, the corresponding 
smoothed processes approximate the original process as A — )• +00, i.e.. Condition (A) is satisfied. 
However, the distance of the set of these ideal low-pass filters from the set of all causal filters is 
positive O. 
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For x(-) G L2(R), we denote by X = Tx the function defined on zR as the Fourier transform 



Here i = \/— T. For x(-) € L2(R), the Fourier transform X is defined as an element of L2(R) 
(more precisely, X{i-) G L2(R))- 

Consider a reference family of "ideal" smoothing filters with the frequency response 



For these filters, Condition (A) is satisfied as ^ — )• 0, and Conditions (B) is satisfied for all fi > 0. 
However, these filters are non-causal: for any x(-) G L2(R), the output processes of these filters 
are weakly predictable at time t on a finite horizon [t. t + /x) |[6l. 

To satisfy Conditions (A)-(D), we consider a family of causal filters with impulse responses 
{hu{-)}'i^i C L2(R) and with the corresponding Fourier transforms Hy{iu}), such that the fol- 
lowing more special Conditions (a)-(d) are satisfied. 

(a) Approximation of identity operator: 

(al) For any > 0, Hy{iu}) — )• 1 as — )• +oo uniformly in cj G [—^, 
(a2) For any x(-) G ^2(1^)5 

- a;(-)llL2(R) ^ as i/^cx), 

where is the output process 



of x{-); 




l-i > 0. 



(1) 




(b) Smoothing property: For every > 0, there exists p > such that for any n > 1, 
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(c) Sub-ideal smoothing: For any 6 > 1, there exists u > such that for any > 

\loR\H,Aioj)\\^ 



{u: |w|>n} 1 + a;2 



-djjj = +00. (2) 



(d) Approximation of non-causal filters ([7]) with respect to the effectiveness in damping: For any 
e > and ^ > 0, there exists u = u{fi) > such that 



\H,{iuj)\-\M^{iuj)\\\^^^^^<e. 



Let us show that Conditions (a)-(d) ensure that Conditions (A)-(D) are satisfied, in a certain 
sense. Clearly, Condition (a) ensures that Condition (A) is satisfied. 
Further, by Condition (b), for any A; > and > 0, 



/oo 
{l + \uj\'')^\H^{iuj)\^duj < +00. 
-oo 



Let x(-) G L2(R), X = TX, and Y,^{iuj) = H^{iuj)X{iuj). By Holder inequality, it follows that 

/oo / poo \ 1/2 

(1 + \u\''f\Y,{iu;)\^du; < / (1 + \u:\'')^\H,{iutdu ll^lli^'a) < +oo. 
-co \J —OD J 

Hence yvif) has derivatives in L2(R) of any order, and, therefore, is infinitely differentiable in the 
classical sense. Therefore, Condition (b) ensures that Condition (B) is satisfied. 

Let us show that Condition (c) ensures that Condition (C) is satisfied. Let 5 > 1 be fixed, 
and let 1/ = 1/(5) be such that © holds. Let us show that the filter with the frequency response 
h = T~^H cannot be causal for some "better" frequency response H{iuj) such that 

\H{iu})\ = o{\H^{iu})\) as +oo. (3) 

More precisely, we will show that h cannot be causal with a stronger condition that there exists 
> such that 

\log\H{iuj)\\>\log\H^{iuj)\\^, \uj\>n. (4) 



In particular, this condition implies that log \Hi,{iuj)\/ log \H{iuj)\ — )• as |a;| — )• +00. 

The desired fact that h cannot be causal can be seen from the following. By Paley and Wiener 
Theorem |[T4l . the Fourier transform H{iuj) of a causal impulse response /i(-) G -^^2(1^) has to be 
such that 



(see, e.g., lfT3l . p. 35). Since v = ^{S) is such that ^ holds, it follows from (01) that h cannot be 
causal. Therefore, Condition (c) ensures that Condition (C) is satisfied. 

Finally, Condition (d) ensures that Condition (D) is satisfied, since the effectiveness of smooth- 
ing is defined by the rate of damping of the higher frequencies. 

3 A family of sub-ideal smoothing filters 

Let C"*" = {z G C : Re 2; > 0}. Let us consider a set of transfer functions 



Here a > 0, 6 > 0, and 5 G [g, 1), are rational numbers, g G (0, 1) is a given number. We mean 
the branch of [s + 6)"^ such that its argument is gArg {s + b), where Arg z G {—it, vr] denotes the 
principal value of the argument of z G C. This set was introduced in H as an auxiliary tool for 
solution of a parabolic equation in the frequency domain. 

Let us consider the set of all transfer functions ^ with rational numbers a > 0, 6 > 0, and 
q G [1/2,1). We assume that this countable set is counted as a sequence {H,^}'^^^ such that 
a— i>0, 6— i>0, g— i>lasz^— )• +00. 

Theorem 1 Conditions (a)-(d) are satisfied for the family of filters defined by the transfer func- 
tions {Hy}'^^^. (Therefore, Conditions (A)-(D) are satisfied for this family ). 




Ha,h,q{s) = e 



a{s+h)i 



s G C 



1+ 



(5) 
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Proof of TheoremU} Let be the Hardy space of holomorphic on C+ functions h{p) with 
finite norm \\h\\Hr = sup^>o + ^'^)IIl,.{R)' ^ ^ (see, e.g., Q). 

Clearly, the functions Hy(jp) are holomorphic in C+, and 

In \H^{s)\ = -Re (a(s + b)i) = -a\s + ft]" cos[gArg (s + h)]. (6) 

In addition, there exists M = M {b, q) > such that cos[gArg {p + b)] > M for all s € C+. It 
follows that 

\H^{s)\ < e-'^^^l^+^l' < 1, s G C+. (7) 

Hence G ff^' for all r G [1, +oo]. By Paley -Wiener Theorem, the inverse Fourier transforms 
h,^ = F^^Hy{iuj) are causal impulse responses, i.e., hy{t) = for t < (see, e.g., fT9l, p. 163). 
Let X e L2(R), X = Fx, and = H^X. 

Let us show that Condition (a) holds. Since a — as — )• +oo, it follows that Hy{ioj) — 1 
as — +00 for any oj and that Condition (al) holds. By Condition (al), Yi,{iijj) — X{iu}) as 
u — ;> +00 for all w G R. In addition, \Hu{iuj)\ < 1. Hence \Yt,{iuj) — X{iuj)\ < 2\X{iuj)\. We 
have that ||^(ii^)||L2(R) — II^IIl2(R) < +oo. By Lebesgue Dominance Theorem, it follows that 

\\Y^{iuj)-X{ 

Therefore, Condition (a) holds. 

Let us show that Condition (b) holds. By i^, it follows that 

\H^(.iuj)\ < e"'^*^!'^!', uj e K. (8) 

Therefore, Condition (b) holds with any p < q. 

To see that Condition (c) holds, it suffices to observe that Q holds if 6q > 1, i.e., q > 1/6. 

Let us show that Condition (d) holds. We assume that the family of transfer functions Ha^b,q{') 
is counted as a sequence {Hx}'^^^ such that 6— )-0, g— )-lasA— )• +oo, with a = fj,/ cos((;7r/2). 
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We have that cos[gArg {ico + b)] > cos{qir/2) for all 6 > 0, a; G R and cos[gArg {iui + b)] — )• 
cos(g7r/2) as 6 — )■ for all g < 1, w G R. 

By (O, IffAl^"^)! — ^ l-^Ai(^'^)l as A — ^ +oo for all a; G R. Further, we have that 

- ln\Hx{iuj)\ = aRe [{iuj + b^] = a\iuj + b\'' coslqArg {iu + b)] 



Hence - In \Hx{iuj)\ > fi\uj\'' and \Hx{iuj)\ < e"^!'^!'. Since q > q > 0,we have from dH) that, 
for some constants ci > and C2 > 0, |//a(^'^)| + \Ma{iuj)\ < c\ exp (— C2(|f:ij|'^ + I^^D) for all 
w. By Lebesgue Dominance Theorem, it follows that 



Hence Condition (d) holds. This completes the proof of Theorem [T] □ 

Note that the sequence Hx{s) introduced above does not ensure approximation described by 
Condition (al), since a — +oo and Hx{0) — as A — +oo. On the other hand, the sequence 
Hy{s) does not ensure approximation (ITOl ). The following corollary shows a way toward the 
combination of these approximation properties. 

Corollary 1 Let q G \q, 1) and b > be given. Let a sequence {//„(•)} = {Ha^b,q{')} be selected 
such that a — )• 0. Then Condition (al) holds for this sequence, and \ Hn{ioj)\ < e^^''^''' for all uj, 
where c = acos(g7r/2). 

Proof of Corollary [T] follows from 

4 Illustrative examples 

The sequence {Hx{-)} introduced in the proof above is such that || |i/;^(ia;)| — 
exp(— ^|a;|)||j;^2(R) — > as A — )■ +cxd, i.e., it approximates the gain of the non-causal smooth- 
ing filter with the frequency response M^{iuj) = exp(— ^|a;|). This sequence corresponds to a 



> a\iuj + cos(g7r/2) > a\uj\'' cos((77r/2). 



(9) 




as A — +00. 



(10) 
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sequence {Ha^b,q{-)} soch that q ^ 1, b ^ 0, a = fx / cos{q7r /2) . Figure [J shows the shapes 
of gain curves \M^{iio)\ = exp(— /i|a;|) for the reference non-causal filter with /u = 0.1 and 

6 q(ia;)| for sub-ideal causal filters ^ with q = 0.99 and b = I — q = 0.01 and q = 0.9, 
b = 1 — q = O.l respectively. In both cases, a = fi/ cos(g7r/2) was used. As expected, damping 
on higher frequencies is more effective for the non-causal filter than for causal ones, and is more 
effective for q = 0.99 than for q = 0.9. It can be illustrated as the following: for uj = 1000, the 
ratio '^X/I^'l'/^'^^' is found to be 1.47 and 38.65 for q = 0.99 and q = 0.9 respectively. 

Figure |2] illustrates Corollary [T] and shows the shapes of error curves for approximation of 
identity operator on low frequencies. More precisely, it shows |M^(i(^) — 1| = |e^^l'^l — l| for 
the reference non-causal filter with fi = 0.05 and \Hafi,q{iio) — 1| for sub-ideal causal filters Q 
with a = b = 0.1 and a = b = 0.05 respectively, with q = 0.5. 

Figure [3] shows an example of impulse response ha,b,q{t) = Ha^h,q){t) calculated as the 
inverse Fourier transform for causal filter ^ with q = 0.9, b = 0.1, a = 1/ cos(g7r/2) = 6.3925. 
It can be seen that the impulse response function almost vanishes on some interval near zero, i.e., 
it is close to a causal impulse response with delay. (However, it does not become a response with 
delay). There is a reason for this: if g — 1 and 6 — then, for a given a, c > 0, Ha^b,q{p) ^""^ 
uniformly in the domain {p G C+ : |p| < c}. 

It can be noted that the phase shift for the frequency response function is large for large 
and it is increasing when g — )• 1. This does not affect much the performance of the filter since the 
gain is small for these large |aj| and Condition (al) is ensured. 

5 Conclusion 

The paper proposes a family of causal smoothing filters with almost exponential damping of the 
energy on the higher frequencies and with the frequency response that can be selected to be arbi- 
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trarily close to the real unity uniformly on an arbitrarily large interval. These filters are sub-ideal 
meaning that a faster decay of the frequency response would lead to the loss of causahty; this is 
because they approximate non-causal filters with exponential rate of decay. A possible appUcation 
is in interpolation and forecast algorithms. The transfer functions obtained are not rational func- 
tions; it would be interesting to consider their approximation by the rational functions. Another 
problem is the transition to discrete time processes. We leave it for future research. 
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Figure 1: Gain decay: shapes of \M^{iuj)\ = exp(— /i|a;|) for non-causal filter with fi = 0.1 and 
\Hafi^q{iuj)\ for causal filters ([5]) with q = 0.99, b = 0.01 and q = 0.9, 6 = 0.1 respectively, with 
a = n/ cos((77r/2). 
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Figure 2: Approximation of identity operator: shapes of error curves \M^{iuj) — 1| and 
\Hafi,q{iuj) — 1| respectively for non-causal filter with /x = 0.05 and for causal filters ^ with 
a = 6 = 0.1 and a = b = 0.05, with q = 0.5. 
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Figure 3: Impulse response hafi^q{t) = {F ^Ha^h^q){t) for causal filter ^ with q = 0.9, b = 0.1, 
a = l/cos(g7r/2) = 6.3925. 
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